[00:01.520 --> 00:14.320]  Now we have some audio. Okay, so everyone somewhat speak?
[00:22.950 --> 00:25.910]  Okay, now try again. Yeah, now we have audio.
[00:26.150 --> 00:28.810]  Hello, good. Can you hear me, Sven?
[00:29.310 --> 00:30.990]  I can hear you. Hey, Irem.
[00:31.150 --> 00:35.530]  Oh my gosh, it was just a simple Windows reboot fixes everything.
[00:36.910 --> 00:41.130]  Hey, I unfortunately, I actually have a meeting right now. So I'm going to drop and join again.
[00:41.130 --> 00:44.510]  What time do I need to be on for speaker time?
[00:45.010 --> 00:46.510]  In 30 minutes.
[00:47.510 --> 00:50.630]  But only in Discord or also in Zoom?
[00:51.110 --> 00:52.330]  Only in Discord.
[00:53.310 --> 00:54.190]  Okay.
[00:56.070 --> 01:00.810]  Yep, so we've got people in the Slack chat or in the Twitch chat saying they can hear us.
[01:00.810 --> 01:01.950]  So I think we're good to go.
[01:02.330 --> 01:03.110]  Sweet.
[01:04.070 --> 01:07.310]  Well, hi, everyone. Sorry about the audio issues.
[01:07.310 --> 01:10.430]  That seems to be the start of every stream is figuring out the audio.
[01:11.790 --> 01:14.910]  The only people I've seen get it right are like Critical Role.
[01:17.730 --> 01:20.290]  They pay a lot of people to get it right.
[01:20.490 --> 01:26.910]  Yeah. Welcome to AI Village at DEF CON 28 safe mode.
[01:28.010 --> 01:29.990]  We're all doing this from home.
[01:32.750 --> 01:36.930]  I hope that this can be as fulfilling to you guys as it is to us.
[01:38.690 --> 01:46.430]  And we just want to start off the village with some like general questions about the
[01:46.430 --> 01:53.730]  state of the field as it is right now at the end, halfway through 2020.
[01:54.430 --> 01:59.710]  So Jag has got a bunch of questions prepared and the rest of us are going to try to answer them.
[02:01.090 --> 02:02.010]  Okay.
[02:02.890 --> 02:04.370]  Welcome, everyone. Thanks.
[02:04.370 --> 02:05.890]  Thank you for being here.
[02:05.890 --> 02:09.510]  The first question that I have listed is...
[02:09.510 --> 02:13.430]  Let's just knock this one out of the way real first.
[02:13.670 --> 02:17.610]  Facial recognition is a security concern for us.
[02:18.110 --> 02:20.490]  In the ML field, of course.
[02:21.090 --> 02:24.070]  And before you answer, please introduce yourself.
[02:24.750 --> 02:27.950]  Yeah. We should probably do a who is who.
[02:29.530 --> 02:31.990]  Sven, do you want to kick us off?
[02:33.050 --> 02:33.990]  Sure.
[02:33.990 --> 02:35.210]  So I'm Sven.
[02:36.110 --> 02:39.850]  I am a senior data scientist at Elastic.
[02:39.850 --> 02:46.870]  Before that, I was a topologist who accidentally got into machine learning and security.
[02:47.850 --> 02:52.370]  And I kind of helped run the AI village.
[02:55.010 --> 02:57.030]  John, do you want to go next?
[02:57.230 --> 02:58.070]  Yeah, sure.
[02:58.070 --> 02:58.790]  Hey, everyone.
[02:58.790 --> 02:59.770]  DeltaZero here.
[02:59.770 --> 03:02.770]  I do data science for intrusion detection.
[03:02.770 --> 03:07.770]  Yeah, I've been in the field for maybe, I don't know, four years now.
[03:07.770 --> 03:09.890]  But pretty awesome.
[03:09.890 --> 03:11.110]  Glad to be here.
[03:14.110 --> 03:15.270]  Rich?
[03:16.170 --> 03:17.910]  Hi, I'm Rich Huerang.
[03:17.970 --> 03:21.690]  I'm a director of data science research at Sophos.
[03:21.690 --> 03:28.310]  I've been working at the intersection of machine learning, security, and privacy for probably
[03:28.310 --> 03:30.250]  about 10 years now.
[03:30.250 --> 03:32.250]  Got my PhD from UCSB.
[03:32.250 --> 03:37.750]  Spent some time with the US Army Research Laboratory, and I've been at Invencion Sophos
[03:37.750 --> 03:39.050]  ever since.
[03:44.460 --> 03:45.600]  Yaga?
[03:46.040 --> 03:47.740]  Hi, I'm Yaga.
[03:47.740 --> 03:52.280]  I am a professor at a small university in Virginia.
[03:52.520 --> 04:02.080]  I've got about 25 years of experience in security, but relatively new to machine learning.
[04:02.080 --> 04:10.180]  My research is focused on securing machine learning, and I like playing video games too.
[04:10.180 --> 04:13.940]  So there's something interesting about it.
[04:14.200 --> 04:14.930]  Anita?
[04:15.640 --> 04:17.080]  Hi, I'm Anita.
[04:17.080 --> 04:19.360]  I'm a research scientist at University of Illinois.
[04:19.360 --> 04:21.500]  Started life as a cryptographer.
[04:21.500 --> 04:24.440]  Been in security for almost 30 years.
[04:28.200 --> 04:31.120]  Okay, and Will's the last guy.
[04:32.360 --> 04:34.400]  Certainly not least.
[04:34.400 --> 04:35.720]  Yeah.
[04:36.560 --> 04:39.280]  Sorry, my morning's just starting.
[04:39.380 --> 04:50.800]  Will, I am a network operator, and I guess attack machine learning models and use AI
[04:50.800 --> 04:53.260]  for offensive purposes.
[04:53.260 --> 04:56.600]  Phishing, pathfinding, and stuff.
[04:58.360 --> 04:59.760]  Yeah, Will's being modest.
[04:59.760 --> 05:05.860]  He actually has the first CVE ever assigned against a machine learning system.
[05:05.860 --> 05:11.240]  If you attend my lab later, there'll be another one.
[05:13.900 --> 05:16.960]  I think that's a challenge for everyone.
[05:18.540 --> 05:22.000]  Can you get one file before Will drops number two?
[05:25.560 --> 05:28.460]  Okay, thank you everyone for the introductions.
[05:28.460 --> 05:30.820]  Sorry, I should have thought about that.
[05:31.120 --> 05:35.740]  Please be sure to ask any questions over in the Discord server.
[05:35.740 --> 05:38.320]  There is a talks channel.
[05:40.800 --> 05:43.420]  Yes, there's a channel for talks.
[05:43.420 --> 05:48.000]  You can ask in there or in the general, and we'll try to catch it.
[05:48.100 --> 05:50.420]  So back to the first question.
[05:50.620 --> 05:56.100]  This was one that was both easy and hard to throw out there.
[05:56.100 --> 06:06.100]  But is facial recognition a security concern when we're talking about privacy?
[06:06.800 --> 06:11.000]  For sure we know about, but in the general machine learning terms,
[06:11.000 --> 06:13.500]  is facial recognition a security issue?
[06:14.880 --> 06:18.960]  So I think just to make the point in our village, one of the things we're trying to do
[06:19.540 --> 06:24.320]  is expand from just thinking of this as a technical kind of algorithm problem.
[06:24.320 --> 06:28.860]  If you look at the broader scope of facial recognition, there are issues not just with
[06:28.860 --> 06:33.040]  security, with data sets, with data integrity, with these kind of choices.
[06:33.440 --> 06:36.480]  So I think my answer would be it is a security issue.
[06:36.480 --> 06:37.540]  It's a machine learning issue.
[06:37.540 --> 06:38.200]  It's a data issue.
[06:38.200 --> 06:39.480]  It's a human issue.
[06:40.280 --> 06:45.000]  And for us, we need to think about all those things as technology people.
[06:47.180 --> 06:51.980]  Yeah, and just to add to that, a lot of the issues around facial recognition
[06:52.980 --> 06:58.520]  are less to do with the technology and more to do with the processes that that technology is
[06:58.520 --> 07:00.480]  embedded in.
[07:00.480 --> 07:06.220]  So if you have a facial recognition system that works, you have issues around privacy, right?
[07:06.220 --> 07:07.840]  Who gets access to that system?
[07:07.840 --> 07:10.560]  Who gets to know who was seen where on which camera?
[07:10.700 --> 07:13.740]  And then when it doesn't work, you have bias issues that sneak in.
[07:13.740 --> 07:16.360]  So how do you control for false positives, right?
[07:16.360 --> 07:22.140]  Someone gets reported as being, you know, facially recognized as a bank robber or something
[07:22.940 --> 07:24.920]  when it's just like an average person.
[07:24.920 --> 07:28.000]  And this actually extends beyond issues of just facial recognition.
[07:28.000 --> 07:30.880]  Just computer vision in general has problems like this.
[07:30.880 --> 07:36.020]  There was a story recently about a license plate recognition system, which led to someone
[07:36.020 --> 07:40.460]  being pulled over and detained, you know, face down on the side of the road for hours
[07:40.460 --> 07:44.180]  before they figured out that it was the wrong license plate number, the wrong make of car,
[07:44.180 --> 07:47.000]  the wrong model year, et cetera, et cetera, et cetera.
[07:47.000 --> 07:53.360]  So yeah, it's definitely a case of not only does the ML work, but do you have the right
[07:53.360 --> 07:57.940]  controls around the ML to make it actually usable?
[07:57.940 --> 08:02.180]  And have you, you know, assessed what happens when it actually fails so that it can fail
[08:02.180 --> 08:04.020]  gracefully without hurting people?
[08:05.560 --> 08:06.840]  Anyone else?
[08:07.320 --> 08:11.540]  I think the last thing, not to dominate this, but the last thing to throw in is I feel like
[08:11.540 --> 08:17.560]  in our village, part of our job is to also translate all this technobabble sometimes
[08:17.560 --> 08:20.800]  to policy speak and to real world speak.
[08:20.800 --> 08:22.800]  And that could be very challenging.
[08:22.800 --> 08:24.820]  We know the technical parts.
[08:24.820 --> 08:29.000]  What ends up on the other end, we need to bridge that gap.
[08:29.680 --> 08:35.920]  I think that's important to understand that security isn't just the technical side.
[08:35.920 --> 08:39.220]  There's a human side to it too, and that extends to policy.
[08:39.220 --> 08:41.140]  So definitely something to think about.
[08:41.140 --> 08:41.520]  Thank you.
[08:42.200 --> 08:48.740]  Staying along the same lines, one of the things that I wanted to ask about is the FOX program.
[08:48.740 --> 08:50.620]  Have you heard about it?
[08:50.620 --> 09:04.180]  And how does it affect our ability for the end user to take back some of that control
[09:04.940 --> 09:06.440]  over their data?
[09:08.560 --> 09:14.140]  It's technically, again, this is, it's the gap between technical aspects of it and
[09:14.140 --> 09:16.380]  implementation and social aspects of it.
[09:16.380 --> 09:18.700]  Technically, it's a brilliant idea.
[09:18.700 --> 09:21.760]  And, you know, it does what it says on the tin.
[09:21.760 --> 09:27.780]  In practice, there are so many ways that people get registered into facial recognition systems
[09:27.780 --> 09:34.320]  from, you know, collecting ID cards to photos that they uploaded before FOX was a big thing
[09:34.320 --> 09:43.520]  to photos that other people take of them and maybe tag them in later, that you really can't
[09:43.520 --> 09:49.480]  just find a quick technical band-aid for security problems that are really a combination of
[09:49.480 --> 09:51.260]  social and technical problems.
[09:51.260 --> 09:55.240]  The root cause here isn't like, oh, we just need better technology to fix this.
[09:55.240 --> 10:01.880]  The root cause is society and sort of laws and norms haven't kept up with the technology.
[10:01.880 --> 10:05.200]  And that's the angle that we need to address it from.
[10:05.200 --> 10:09.060]  Technical band-aids are only ever going to be available to people that know about them,
[10:09.060 --> 10:13.820]  know how to apply them, and can get over that barrier to entry.
[10:13.920 --> 10:17.360]  Really, we need to be thinking broader about impact to society as a whole.
[10:18.160 --> 10:23.360]  It makes me think a little bit of this field of, you know, usable privacy and, you know,
[10:23.360 --> 10:24.980]  privacy by design.
[10:24.980 --> 10:32.700]  See, that's analogous here because people don't understand that long-term impact of
[10:33.300 --> 10:34.540]  having a photo op.
[10:34.540 --> 10:36.880]  So, enabling people to use this tool is great.
[10:36.880 --> 10:41.100]  I think the average person isn't going to even think about it because it doesn't affect
[10:41.100 --> 10:42.460]  their day-to-day life.
[10:42.460 --> 10:48.980]  So, again, I think the issue is, how do you express to them the long-term potential damage
[10:48.980 --> 10:51.280]  of having their pictures out there?
[10:52.140 --> 10:57.460]  Okay, we're going to move into a different arena.
[10:58.320 --> 11:06.680]  What do you say the latest advances that your typical security team could use
[11:08.280 --> 11:10.360]  in terms of machine learning?
[11:11.380 --> 11:16.040]  You know, a lot of vendors are out there saying that this is going to be the panacea for them
[11:16.040 --> 11:17.880]  to be able to fix things.
[11:17.880 --> 11:24.600]  Some of the actual products that will help them or techniques that will help them.
[11:26.440 --> 11:31.940]  I just want to take this opportunity to plug the panel that we have coming up on Saturday
[11:31.940 --> 11:35.540]  on whether or not ML and InfoSec lives up to the hype.
[11:35.540 --> 11:39.640]  So, we're going to be diving, hopefully, pretty deep into those questions.
[11:39.640 --> 11:42.680]  But I'll kick it over to someone else since I've been talking a lot.
[11:47.220 --> 11:52.500]  I really don't know what the boots on the ground could be using.
[11:53.320 --> 11:57.860]  But there are some promising things coming down the pipe from researchers.
[11:59.560 --> 12:00.680]  But I...
[12:02.540 --> 12:05.960]  So, I'll jump in real quick. Sorry to dominate again.
[12:05.960 --> 12:08.600]  And I'll let Will speak because he's more an operator.
[12:08.720 --> 12:13.600]  For me, federated learning, federated and privacy-preserving machine learning is very
[12:13.600 --> 12:15.520]  important for a bunch of reasons.
[12:16.020 --> 12:20.440]  Not the least of which is being able to learn across edge devices that are in multiple disparate
[12:20.440 --> 12:22.740]  places and being able to do it more efficiently.
[12:22.760 --> 12:27.840]  And being able to use new types of data because there's privacy-preserving ways to do it.
[12:27.840 --> 12:33.320]  So, I think I see federated learning as something that's promising.
[12:34.780 --> 12:40.040]  Yeah, I'd say from a product standpoint, it's just like the simple stuff.
[12:40.040 --> 12:49.780]  So, you know, on a network, clients will still miss password spraying internally.
[12:50.020 --> 12:54.520]  And that's just something that shouldn't be happening with the logs that are available.
[12:54.520 --> 13:02.420]  So, I think in terms of just instant response and stuff on networks, it's the simple stuff
[13:02.420 --> 13:06.540]  that is going to give you the most bang for the buck.
[13:09.120 --> 13:14.740]  Obviously, there's a lot of competition in the machine learning space, or it would seem.
[13:17.440 --> 13:24.140]  And half of me, at least 40% of me, thinks in two years, we're going to be moving on
[13:24.140 --> 13:28.160]  from machine learning as a solution in networks.
[13:28.160 --> 13:32.700]  And we'll probably replace it with like zero trust or something like that.
[13:34.080 --> 13:39.120]  But there's also the other 10% that thinks like it will always have a place.
[13:40.020 --> 13:47.980]  But just like with the facial recognition stuff and the bias, if we don't put it out
[13:50.680 --> 13:55.480]  in a way that's useful, and in a way that people can understand,
[13:55.480 --> 14:01.100]  the minute it breaks, they're likely to just throw it out the window and say it doesn't work.
[14:02.700 --> 14:09.820]  Even though, you know, it is a perfectly viable solution. But yeah, I'd say simple stuff.
[14:10.320 --> 14:11.580]  Yeah. Okay.
[14:11.820 --> 14:15.600]  To me, that's the more important side of the question is like,
[14:15.600 --> 14:20.180]  practitioners on the ground, we don't need any new math or anything new interesting.
[14:20.180 --> 14:24.660]  But there's some basics that they can get their hands on that will probably help them out when
[14:24.660 --> 14:30.100]  they have to deal with a deluge of alerts from some sim that they have to deal with.
[14:30.100 --> 14:32.680]  A little bit of clustering could go a long way in that situation.
[14:34.020 --> 14:36.440]  Yeah, I think that's perfect.
[14:36.660 --> 14:41.220]  That's perfect for what I was going to say is, I didn't mean to say products,
[14:41.220 --> 14:47.800]  I probably wrote that down wrong. But like, what can we tell our audience here that maybe
[14:47.800 --> 14:53.860]  practitioners that they can do on their own? So I think something like our workshops,
[14:53.860 --> 14:56.420]  if you take a look at them, they'll be able to help you
[14:56.420 --> 15:00.840]  see that you may not need to go get these expensive products. There's some simple things
[15:00.840 --> 15:04.520]  you can do. Yeah.
[15:04.620 --> 15:09.100]  Yeah, and I just add to that, but from my own experience as someone who's
[15:09.700 --> 15:15.180]  used ML to tackle a lot of security problems, very often you don't even need like the big
[15:15.180 --> 15:20.920]  complicated deep neural networks or gradient boosted decision trees or pick your flavor of
[15:20.920 --> 15:27.200]  the day. Very often, it's enough to have a good idea of the problem you're trying to solve and
[15:27.280 --> 15:31.760]  a good set of representative data and then simple methods like, you know, linear logistic
[15:31.760 --> 15:36.740]  regression will often be enough to tackle it for you. So, you know, you don't need to think that
[15:36.740 --> 15:43.020]  you've got to have tons of like Amazon EC2 credits to run a big beefy GPU machine.
[15:43.020 --> 15:48.020]  A lot of times you can get really like 80% solutions, really good solutions just on your
[15:48.020 --> 15:51.820]  laptop. Okay.
[15:54.940 --> 16:00.380]  For most security to be able to defend yourself, you have to be able to identify
[16:01.000 --> 16:08.920]  the issue or identify the attack. We discuss, you know, adversarial attacks quite a bit
[16:09.520 --> 16:17.320]  in the academic realm. How could someone tell that they're seeing an adversarial attack in practice
[16:18.320 --> 16:26.870]  today? You mean on the AI? I'm throwing out some hard ones. You mean on the AI portion?
[16:29.390 --> 16:36.430]  Well, how could you just notice an attack? Because to defend against something,
[16:36.430 --> 16:40.330]  you typically have to be able to define what that attack looks like, right?
[16:40.930 --> 16:44.970]  At least to some degree. Can we do that today?
[16:47.350 --> 16:52.570]  It's kind of too bad Hiram's not here because I feel like he'd have a lot to say on this topic.
[16:55.410 --> 16:56.770]  Let's hear it.
[16:57.510 --> 17:04.690]  So for the traditional adversarial attacks, we basically are trying to modify an image
[17:04.690 --> 17:09.330]  or something slightly to bypass the machine learning system. That doesn't really work
[17:09.330 --> 17:14.150]  with security because our data is much more structured. You can't like do a gradient descent
[17:14.150 --> 17:20.570]  on a binary and then hope it still compiles or runs because you're going to get byte codes that
[17:20.570 --> 17:28.570]  are just wrong. So I don't think like adversarial attacks as academics that you read up in a big
[17:28.570 --> 17:33.230]  year paper are going to really apply. There are some modifications to that that might apply that
[17:33.230 --> 17:40.370]  might be very helpful. But what we do see already and have been seeing for like a decade is people
[17:40.370 --> 17:46.090]  just trying a bunch of stuff and then they get past it. And the more they get informed they are
[17:46.090 --> 17:51.710]  about how the machine learning model works, the more effective and more like gradient descent it
[17:51.710 --> 17:58.610]  is. You know, like the more effective they are. Like for like the spam filters from back in 2001
[17:58.610 --> 18:06.290]  when people were packing Wikipedia articles at the bottom of their spam messages to bypass the
[18:06.290 --> 18:10.050]  naive Bayesian classifiers, they understood that it wasn't a naive Bayesian classifier. They
[18:10.050 --> 18:16.270]  understood that they needed to pack the thing. And then they did that. And that's going to happen
[18:16.270 --> 18:27.590]  with next-gen AV products too. It may already be happening. I mean, that was what we saw in
[18:27.590 --> 18:35.430]  the silence bypass was effectively a stuffing attack. Yeah. That was researchers being nice to
[18:35.430 --> 18:46.730]  us and telling us about the problem that's happening. So this one is more directed at Will,
[18:46.730 --> 18:51.650]  but definitely want everyone's opinion. And it gets back to something we talked about in a journal
[18:51.650 --> 18:57.710]  club, which for those of you that might not have seen it, we may be replaying it during the weekend.
[18:59.230 --> 19:07.470]  Will received a CVE for the vulnerability he discovered.
[19:07.470 --> 19:17.110]  But do we need actual CVEs specifically for machine learning or should we keep them more general?
[19:19.090 --> 19:25.990]  So I guess first I'd like to say it was our, so Nick Landers and I did it.
[19:26.550 --> 19:34.550]  I just talk about it more so. So the CVE system is designed to communicate risk.
[19:35.310 --> 19:43.330]  And I think anywhere that risk is present, then a CVE is appropriate.
[19:46.150 --> 19:51.510]  You know, I just, I submitted it. So I'm not the one that gets to say whether or not it's a CVE.
[19:52.270 --> 19:59.170]  So I guess that the question would be for the NIST. And I know they've come out with a paper
[19:59.170 --> 20:05.910]  that kind of highlights machine learning attacks or there's a taxonomy. It's like 15 taxonomies
[20:05.910 --> 20:15.470]  right now. But yeah, I think in any way there's risk that is at least measurable, a CVE would
[20:15.470 --> 20:21.310]  be appropriate or any standardized score. I know there's operators like...
[20:22.130 --> 20:29.630]  And I'm sorry, I don't mean should it be a CVE, but more should there be a class just for machine
[20:29.630 --> 20:35.810]  learning, you know, because they break it down. Yeah, yeah, I think so. It's fundamentally
[20:35.810 --> 20:43.390]  different. You know, so for example, like with a traditional exploit, you know, there's like
[20:44.170 --> 20:51.750]  a patch and then you can push it out. But with, for example, like with Proofpoint's model,
[20:51.750 --> 20:58.270]  there's some residual, there's some like lingering theft going on. So
[20:58.990 --> 21:05.230]  once you have that copycat model, you're never going to get 100% of their model. But
[21:06.590 --> 21:13.250]  for some, there's some decay into the future, but you'll have some portion of that model.
[21:13.250 --> 21:20.910]  Your copycat model will be right, you know, up to some future time until they've retrained enough.
[21:20.910 --> 21:28.930]  But in their response, you know, unless they do things to fix it or prevent, you know,
[21:28.930 --> 21:36.610]  people from stealing outputs, then you can always create a new model. So I think that
[21:37.210 --> 21:45.050]  piece is important. But yeah, it's a little, it's fundamentally different beast than
[21:46.610 --> 21:50.990]  than like an exploit. Yeah, I don't know, you guys probably have a better idea of like,
[21:50.990 --> 21:56.550]  how you're supposed to patch a machine learning model or you take it out. I don't know.
[21:57.510 --> 22:00.250]  Long and expensive retraining process.
[22:01.130 --> 22:07.210]  Do you guys agree, though, that it's good to have that separate machine learning
[22:07.210 --> 22:13.550]  kind of category to highlight the fact that this is a vulnerability, a specific type of vulnerability?
[22:15.270 --> 22:22.450]  Yeah, I think most, so there are some exceptions. And, you know, again, I'm going to sort of like
[22:22.450 --> 22:28.890]  promote the Journal Club on the Summoning Demons paper later this weekend. But most machine learning
[22:28.890 --> 22:35.250]  exploits don't translate into exploits as we usually think of them in InfoSec. They are much
[22:35.250 --> 22:42.450]  more either model stealing or model bypass, or maybe like a denial of service type thing. So
[22:43.570 --> 22:48.170]  in my mind, keep in mind, I'm coming at this from the ML side of things and much less from the
[22:48.170 --> 22:53.130]  InfoSec side, I do think it makes a lot of sense to have separate categories that reflect the
[22:53.130 --> 22:59.430]  different kinds of risks that we currently think of ML models as being exposed to.
[22:59.870 --> 23:07.110]  Okay. So that's kind of all the questions I had. The next was, I was going to turn it back over to
[23:07.110 --> 23:12.510]  Sven to kind of give you an overview of the weekend. And thanks for being here, everyone.
[23:12.510 --> 23:18.510]  And thank you for your questions. Or answers. Yeah, cool. I'm really looking forward to the
[23:18.510 --> 23:27.310]  talks. This is going to be a great DEF CON. Cool. I'm out. Later, folks.
[23:27.790 --> 23:35.010]  Have fun. So about this weekend, the way we're structuring it is everyone, all the talks are
[23:35.690 --> 23:40.630]  recorded already and on my desktop. So the speakers won't be talking live. The reason why we're doing
[23:40.630 --> 23:47.090]  that is so that the speakers can focus on answering your questions in Discord. And so during the talk,
[23:47.090 --> 23:52.110]  speakers will be in the Discord and hopefully afterwards. So any questions you have about the
[23:52.110 --> 23:57.850]  thing, you can just ask it in the Discord channel, and we'll answer it. The speaker will answer it,
[23:57.850 --> 24:02.710]  or maybe someone else who's knowledgeable on the subject will help try to answer your question.
[24:05.090 --> 24:12.250]  So we're going to, in a few minutes, launch into the first one, which is we've got Zoltan here.
[24:16.010 --> 24:24.330]  Zahak, who's in Hyrum. They'll be going over to the Discord to answer any questions.
[24:25.830 --> 24:31.230]  And then, yeah, then we've got Eriks and a few others. Oh,
[24:31.810 --> 24:34.670]  Sarah, do you want to introduce yourself, too?
[24:36.710 --> 24:44.770]  Oh, god. SJTurk, ethics chair, so I guess the buck stops here.
[24:46.530 --> 24:50.050]  I guess data nerd who's working on disinformation.
[24:57.600 --> 25:02.100]  I think that's it for the intro stuff.
[25:02.100 --> 25:07.120]  Yeah, that's it. We were trying to leave some time to make sure the transition to Hyrum.
[25:07.650 --> 25:12.080]  Yeah, I've got that all ready to hit play.
[25:12.620 --> 25:19.360]  Okay, yeah. Oh, workshops. Yeah, so there's a few workshops.
[25:20.630 --> 25:27.550]  They're mostly focused on sort of bypasses and interesting techniques on...
[25:28.270 --> 25:39.110]  The hard part for us is text data. So, GtKondike, Gavin Kondike, well, has a lovely workshop on
[25:42.990 --> 25:54.750]  spam builders. And we have... Will here is talking about a new, basically, CVE2.
[25:57.590 --> 25:59.650]  Model stealing stuff.
[26:02.930 --> 26:12.030]  And the way those work are they'll be streamed on Twitch live. They're not recorded. And then
[26:12.030 --> 26:18.790]  there'll be a few people who are TAing who will help the workshop presenter help relay questions
[26:18.790 --> 26:25.170]  from Discord over to the workshop presenter and help resolve any minor issues you guys might have.
[26:26.330 --> 26:31.270]  There are Jupyter notebooks and stuff posted on the website.
[26:33.070 --> 26:41.250]  So, hopefully this all works out. It's a pretty normal machine learning workshop structure.
[26:41.570 --> 26:47.570]  Just online versus in person so that you can't have someone spy over your shoulder to tell you
[26:47.570 --> 26:58.050]  your typo is. Well, with that, let's start Hyrum's...
[27:02.410 --> 27:08.170]  Okay. I'm gonna go ahead and drop off. Hi, everyone. Have a great DevCon.
[27:08.290 --> 27:12.190]  We'll do. Have fun. Awesome. Thanks all.
